Modelling Low Power Compute Clusters for Cloud Simulation

In order to minimise their energy use, data centre operators are constantly exploring new ways to construct computing infrastructures. As low power CPUs, exemplified by ARM-based devices, are becoming increasingly popular, there is a growing trend for the large scale deployment of low power servers in data centres. For example, recent research has shown promising results on constructing small scale data centres using Raspberry Pi (RPi) single-board computers as their building blocks. To enable larger scale experimentation and feasibility studies, cloud simulators could be utilised. Unfortunately, state-of-the-art simulators often need significant modification to include such low power devices as core data centre components. In this paper, we introduce models and extensions to estimate the behaviour of these new components in the DISSECT-CF cloud computing simulator. We show that how a RPi based cloud could be simulated with the use of the new models. We evaluate the precision and behaviour of the implemented models using a Hadoop-based application scenario executed both in real life and simulated clouds

Understanding the Performance of Low Power Raspberry Pi Cloud for Big Data

Nowadays, Internet-of-Things (IoT) devices generate data at high speed and large volume. Often the data require real-time processing to support high system responsiveness which can be supported by localised Cloud and/or Fog computing paradigms. However, there are considerably large deployments of IoT such as sensor networks in remote areas where Internet connectivity is sparse, challenging the localised Cloud and/or Fog computing paradigms. With the advent of the Raspberry Pi, a credit card-sized single board computer, there is a great opportunity to construct low-cost, low-power portable cloud to support real-time data processing next to IoT deployments. In this paper, we extend our previous work on constructing Raspberry Pi Cloud to study its feasibility for real-time big data analytics under realistic application-level workload in both native and virtualised environments. We have extensively tested the performance of a single node Raspberry Pi 2 Model B with httperf and a cluster of 12 nodes with Apache Spark and HDFS (Hadoop Distributed File System). Our results have demonstrated that our portable cloud is useful for supporting real-time big data analytics. On the other hand, our results have also unveiled that overhead for CPU-bound workload in virtualised environment is surprisingly high, at 67.2%. We have found that, for big data applications, the virtualisation overhead is fractional for small jobs but becomes more significant for large jobs, up to 28.6%

SDN-based Virtual Machine Management for Cloud Data Centers

Software-Defined Networking (SDN) is an emerging paradigm to logically centralize the network control plane and automate the configuration of individual network elements. At the same time, in Cloud Data Centers (DCs), even though network and server resources converge over the same infrastructure and typically under a single administrative entity, disjoint control mechanisms are used for their respective management. In this paper, we propose a unified server-network control mechanism for converged ICT environments. We present a SDN-based orchestration framework for live Virtual Machine (VM) management where server hypervisors exploit temporal network information to migrate VMs and minimize the network-wide communication cost of the resulting traffic dynamics. A prototype implementation is presented and Mininet is used to evaluate the impact of diverse orchestration algorithms

Scalable Traffic-Aware Virtual Machine Management for Cloud Data Centers

Virtual Machine (VM) management is a powerful mechanism for providing elastic services over Cloud Data Centers (DC)s. At the same time, the resulting network congestion has been repeatedly reported as the main bottleneck in DCs, even when the overall resource utilization of the infrastructure remains low. However, most current VM management strategies are traffic-agnostic, while the few that are traffic-aware only concern a static initial allocation, ignore bandwidth oversubscription, or do not scale. In this paper we present S-CORE, a scalable VM migration algorithm to dynamically reallocate VMs to servers while minimizing the overall communication footprint of active traffic flows. We formulate the aggregate VM communication as an optimization problem and we then define a novel distributed migration scheme that iteratively adapts to dynamic traffic changes. Through extensive simulation and implementation results, we show that S-CORE achieves significant (up to 87%) communication cost reduction while incurring minimal overhead and downtime. Index Terms—Virtual Machine, Migration, Consolidation, Communication Cost, Scalable, Traffic-Aware, Data Center Networ

SDN-based Virtual Machine Management for Cloud Data Centers

Software-Defined Networking (SDN) is an emerging paradigm to logically centralize the network control plane and automate the configuration of individual network elements. At the same time, in Cloud Data Centers (DCs), although network and server resources are collocated and managed by a single administrative entity, disjoint control mechanisms are used for their respective management. In this article, we propose a unified server-network resource management for such converged Information and Communication Technology (ICT) environments. We present a SDN-based orchestration framework for live Virtual Machine (VM) management that exploits temporal network information to migrate VMs and minimize the network-wide communication cost of the resulting traffic dynamics. A prototype implementation is presented, and a Cloud DC testbed is used to evaluate the impact of diverse orchestration algorithms. Our live VM management has been shown to reduce the network-wide communication cost, especially for the high-cost and congestionprone core and aggregation layers of the DC. Our results show an increase in network-wide throughput by over 6 times, as well as over 70% communication cost reduction by migrating less than 50% of the VMs

Network and Server Resource Management Strategies for Data Centre Infrastructures: A Survey

The advent of virtualisation and the increasing demand for outsourced, elastic compute charged on a pay-as-you-use basis has stimulated the development of large-scale Cloud Data Centres (DCs) housing tens of thousands of computer clusters. Of the signi�cant capital outlay required for building and operating such infrastructures, server and network equipment account for 45% and 15% of the total cost, respectively, making resource utilisation e�ciency paramount in order to increase the operators' Return-on-Investment (RoI). In this paper, we present an extensive survey on the management of server and network resources over virtualised Cloud DC infrastructures, highlighting key concepts and results, and critically discussing their limitations and implications for future research opportunities. We highlight the need for and bene �ts of adaptive resource provisioning that alleviates reliance on static utilisation prediction models and exploits direct measurement of resource utilisation on servers and network nodes. Coupling such distributed measurement with logically-centralised Software De�ned Networking (SDN) principles, we subsequently discuss the challenges and opportunities for converged resource management over converged ICT environments, through unifying control loops to globally orchestrate adaptive and load-sensitive resource provisioning

PLAN: Joint Policy- and Network-Aware VM Management for Cloud Data Centers

Policies play an important role in network configuration and therefore in offering secure and high performance services especially over multi-tenant Cloud Data Center (DC)environments. At the same time, elastic resource provisioning through virtualization often disregards policy requirements, assuming that the policy implementation is handled by the underlying network infrastructure. This can result in policy violations, performance degradation and security vulnerabilities. In this paper, we define PLAN, a PoLicy-Aware and Network-aware VM management scheme to jointly consider DC communication cost reduction through Virtual Machine (VM) migration while meeting network policy requirements. We show that the problem is NP-hard and derive an efficient approximate algorithm to reduce communication cost while adhering to policy constraints. Through extensive evaluation, we show that PLAN can reduce topology-wide communication cost by 38% over diverse aggregate traffic and configuration policies

Performance analysis of single board computer clusters

The past few years have seen significant developments in Single Board Computer (SBC) hardware capabilities. These advances in SBCs translate directly into improvements in SBC clusters. In 2018 an individual SBC has more than four times the performance of a 64-node SBC cluster from 2013. This increase in performance has been accompanied by increases in energy efficiency (GFLOPS/W) and value for money (GFLOPS/$). We present systematic analysis of these metrics for three different SBC clusters composed of Raspberry Pi 3 Model B, Raspberry Pi 3 Model B+ and Odroid C2 nodes respectively. A 16-node SBC cluster can achieve up to 60GFLOPS, running at 80W. We believe that these improvements open new computational opportunities, whether this derives from a decrease in the physical volume required to provide a fixed amount of computation power for a portable cluster; or the amount of compute power that can be installed given a fixed budget in expendable compute scenarios. We also present a new SBC cluster construction form factor named Pi Stack; this has been designed to support edge compute applications rather than the educational use-cases favoured by previous methods. The improvements in SBC cluster performance and construction techniques mean that these SBC clusters are realising their potential as valuable developmental edge compute devices rather than just educational curiosities

Who should be prioritized for renal transplantation?: Analysis of key stakeholder preferences using discrete choice experiments

Background Policies for allocating deceased donor kidneys have recently shifted from allocation based on Human Leucocyte Antigen (HLA) tissue matching in the UK and USA. Newer allocation algorithms incorporate waiting time as a primary factor, and in the UK, young adults are also favoured. However, there is little contemporary UK research on the views of stakeholders in the transplant process to inform future allocation policy. This research project aimed to address this issue. Methods Discrete Choice Experiment (DCE) questionnaires were used to establish priorities for kidney transplantation among different stakeholder groups in the UK. Questionnaires were targeted at patients, carers, donors / relatives of deceased donors, and healthcare professionals. Attributes considered included: waiting time; donor-recipient HLA match; whether a recipient had dependents; diseases affecting life expectancy; and diseases affecting quality of life. Results Responses were obtained from 908 patients (including 98 ethnic minorities); 41 carers; 48 donors / relatives of deceased donors; and 113 healthcare professionals. The patient group demonstrated statistically different preferences for every attribute (i.e. significantly different from zero) so implying that changes in given attributes affected preferences, except when prioritizing those with no rather than moderate diseases affecting quality of life. The attributes valued highly related to waiting time, tissue match, prioritizing those with dependents, and prioritizing those with moderate rather than severe diseases affecting life expectancy. Some preferences differed between healthcare professionals and patients, and ethnic minority and non-ethnic minority patients. Only non-ethnic minority patients and healthcare professionals clearly prioritized those with better tissue matches. Conclusions Our econometric results are broadly supportive of the 2006 shift in UK transplant policy which emphasized prioritizing the young and long waiters. However, our findings suggest the need for a further review in the light of observed differences in preferences amongst ethnic minorities, and also because those with dependents may be a further priority.</p